1. Field of the Invention
The present invention relates to motion estimation and compensation. More particularly, the present invention relates to a method of and apparatus for setting a virtual image region in order to efficiently perform motion estimation and compensation of a panorama image including 360-degree omni-direction image information.
2. Description of the Related Art
An omni-directional video camera system is a camera system that is capable of taking 360-degree omni-directional images from a fixed viewpoint. The omni-directional video camera system takes omni-directional images by mounting and using a special shape mirror such as a hyperboloid mirror, a special lens such as a fish eye lens, or by using a plurality of cameras.
A 3-dimensional realistic broadcasting system is presented as an example of an omni-directional video coding application for use with such a system. In the 3-dimensional realistic broadcasting system, all image information on views from various viewpoints in a baseball game and the like for example, is provided to viewers' terminals. That is, a variety of image information items, including the view from the pitcher, the view from the catcher, the view from the batter, and the view from the audience sitting near first base, are provided to the viewers. Viewers can select a desired viewpoint and view the image from the viewpoint.
An image photographed by the omni-directional camera system has a characteristic corresponding to a 3-dimensional spherical environment. Accordingly, a 3-dimensional image taken by the omni-directional camera system is converted into a 2-dimensional plane image. At this time, the 2-dimensional plane image becomes a panorama image including the omni-directional image, and omni-directional video coding is performed for the 2-dimensional panorama image.
Generally, this panorama image has a size much larger than that of the conventional 2-dimensional image, and in order to transmit a quality image, uses an excessively wide bandwidth. In an omni-directional image, a user is typically not interested in all image parts obtained by the omni-directional camera at the same time, but wants to view only a part of interest in detail by freely manipulating user's viewpoints based on information on the entire image. Accordingly, a codec capable of transmitting an image with a minimum transmission bandwidth to a receiving end is needed, wherein the image is of interest to the current user. In this regard, a method of dividing a panorama image into tiled patches and transmitting the image has been suggested.
FIG. 1 illustrates a method of dividing a panorama image into tiled patches and transmitting the image.
In this method, an encoded panorama image is divided into patches of small tiles and stored, and then, on the request of a user, some patches centered on a part to be shown by the request are transmitted to a decoder side. According to this method, the entire panorama image is not transmitted to the decoder side at one time, but is divided into tiled patches and transmitted such that the size of the data transmitted and the transmission bandwidth are greatly reduced, and the initial delay required for transmitting all image information is prevented. Also, the surrounding part of the area of interest being currently seen by the request of the user is transmitted and received together, such that the user can easily change viewpoints.
In order to transmit the tiled patches to the decoder side, a transmission and reception method using the conventional 2-dimensional moving picture coding techniques without change has been suggested. Techniques such as MPEG-4 Part 2 Visual or H.264 are examples of conventional 2-dimensional moving picture coding techniques being used. As shown in FIG. 2, according to the conventional technology, in order to enhance efficiency when motion estimation or compensation is performed along the boundary of a reference image frame, an unrestricted motion vector (UMV) mode is prepared and a virtual image region is disposed. Then, by padding pixel values of the boundary part below, above, left and right of the frame, motion estimation of a current image is performed. Since any image data other than an image frame to be transmitted cannot be referred to in the 2-dimensional moving picture processing technique, this method is used to obtain high efficiency in the situation. However, in order to encode a large panorama image, an image frame of small-tile-shaped patch units is encoded, and reference images can exist with respect to the position of each patch.
Accordingly, a need exists for a system and method for encoding images more efficiently by using reference images that can exist with respect to the position of each patch, as the conventional moving picture coding methods are not appropriate.